Musical Modality: The Development and Evaluations of Augmented Modality-based Music Systems

of Master’s Thesis Academic Year 2000 Musical Modality: The Development and Evaluations of Augmented Modality-based Music Systems This research aims to propose new music interactive systems to extend a general idea of musical instruments. Speculating on the tactile interaction in music, I built interactive systems that provide new musical modality for a new kind of musical pleasure. In the field of music research, many systems using computers for sounds and music interaction have been introduced. These systems for enrichment of our music expressions can be considered as new experiments to extend existing methods of culture and arts. In this research, I propose three music interactive systems, that provide a feeling of direct manipulation and easy controls of music. These systems aim to enable a beginner to enjoy being performer and audience at the same time. After building mappings of musical elements for each system, I evaluated these systems. At first I extended a first prototype of the “Iamascope” system by redesigning the musical components of the system. Iamascope is an interactive multimedia instrument using image captured by a camera, and it generates kaleidoscopic images and music by the movements of the performer without any tactile interaction. I investigate the musical modality of the music interaction with the non-tactile interface, and propose some guidelines for music interactive systems. Next I propose a novel musical modality with tactile interaction by using fluid media. The musical instrument, “Tangible Sound #2,” uses the water flow both as an input medium and a metaphor of the flowing sounds and music. It is considered that the unique tactile feeling of the water provides a new modality of music interaction. From the evaluation of user interaction, I conclude that the unique sense of touch is an important element for the musical modality in this system. The last system proposes the musical modality which changes itself correspondent with contexts. “Com-Music” is a music expression system which generates music by interaction between the user and a sensor-equipped doll. The doll has five levels of interaction as pre-defined contexts, that have different sets of music control mappings. I evaluate the user interaction of this system to discuss about the effect upon user interaction from the musical modality correspondent with contexts. Afterwords, I discuss the pleasure of musical and sound expression. I explain the new musical modality provided by the extention of musical instruments with the design and evaluation of these three systems.